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WHAT IS CLAIMED IS: 

1 1 . A method comprising: 



2 a) tracking, for a set of one or more ads, ad set performance information 

3 with respect to a document; and 

4 b) aggregating, for the document, tracked ad set performance 

5 information. 



1 2. The method of claim 1 wherein the ad set performance information includes 

2 selection and impression information. 

1 3. The method of claim 1 wherein the document is a Web page identified by a 

2 URL. 



1 4. The method of claim 1 further comprising: 

2 c) thresholding the ad set performance information. 

1 5. The method of claim 1 further comprising: 

2 c) determining a confidence measure of the ad set performance 

3 information; and 

4 d) combining general ad set performance information and the ad set 

5 performance information with respect to the document using the 

6 determined confidence measure. 



1 6. The method of claim 5 wherein the act of determining a confidence measure 

2 uses at least one of a data amount and a data age. 

1 7. A method comprising: 

2 a) tracking, for a set of one or more ads, ad set performance information 

3 with respect to a set of documents, wherein the set of documents is a 

4 subset of a document collection; and 
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5 b) aggregating, for the set of documents, tracked ad set performance 

6 information. 

1 8. The method of claim 7 wherein the ad set performance information includes 

2 selection and impression information. 

1 9. The method of claim 7 wherein the set of documents includes related Web 

2 pages. 

1 10. The method of claim 7 further comprising: 

2 c) thresholding the ad set performance information. 

1 11. The method of claim 7 further comprising: 

2 c) determining a confidence measure of the ad set performance 

3 information; and 

4 d) combining general ad set performance information and the ad set 

5 performance information with respect to the set of documents using the 

6 determined confidence measure. 

1 12. The method of claim 1 1 wherein the act of determining a confidence 

2 measure uses at least one of a data amount and a data age. 

1 13. A method comprising: 

2 a) accepting, for a set of one or more ads, performance information for a 

3 document; and 

4 b) scoring or modifying a score of each of one or more ads using the 

5 accepted ad set performance information for the document. 

1 14. The method of claim 13 wherein the ad set performance information includes 

2 selection and impression information. 
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1 1 5. The method of claim 1 3 wherein the document is a Web page identified by a 

2 URL. 

1 1 6. The method of claim 1 3 wherein the act of scoring or modifying a scoring 

2 includes: 

3 i) determining a first ad score using, at least, general performance 

4 information for the ad set, and 

5 ii) modifying the first ad score using the accepted ad set 

6 performance information for the document. 

1 1 7. The method of claim 1 6 wherein the act of determining a first score further 

2 uses, at least, document information. 

1 1 8. The method of claim 1 6 wherein the act of determining a first score further 

2 uses, at least, ad targeting information. 

1 1 9. The method of claim 1 6 wherein the act of determining a first score further 

2 uses, at least, ad relevance information. 

1 20. The method of claim 13 wherein the act of scoring or modifying a scoring 

2 includes: 

3 i) determining mixed performance information using, at least, 

4 general performance information for the ad set and the accepted ad 

5 set performance information for the document, and 

6 ii) scoring the ad using the determined mixed performance 

7 information. 

1 21 . The method of claim 20 wherein the act scoring the ad further uses, at least, 

2 document information. 
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1 22. The method of claim 20 wherein the act of scoring the ad further uses, at 

2 least, ad targeting information. 

1 23. The method of claim 20 wherein the act of scoring the ad further uses, at 

2 least, ad relevance information. 

1 24. A method comprising: 



2 a) accepting, for a set of one or more ads, performance information for a 

3 set of documents, wherein the set of documents is a subset of a document 

4 collection; and 

5 b) scoring or modifying a score of each of one or more ads using the 

6 accepted ad set performance information for the set of documents. 



1 25. The method of claim 24 wherein the ad set performance information includes 

2 selection and impression information. 

1 26. The method of claim 24 wherein the set of documents includes related Web 

2 pages. 

1 27. The method of claim 24 wherein the act of scoring or modifying a score 

2 includes: 



3 i) determining a first ad score using, at least, general performance 

4 information for the ad set, and 

5 ii) modifying the first ad score using the accepted ad set performance 

6 information for the set of documents. 



1 28. The method of claim 27 wherein the act of determining a first score further 

2 uses, at least, document information. 

1 29. The method of claim 27 wherein the act of determining a first score further 

2 uses, at least, ad targeting information. 
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1 30. The method of claim 27 wherein the act of determining a first score further 

2 uses, at least, ad relevance information. 

1 31 . The method of claim 24 wherein the act of scoring or modifying a score 

2 includes: 



3 i) determining mixed performance information using, at least, 

4 general performance information for the ad set and the accepted ad 

5 set performance information for the set of documents, and 

6 ii) scoring the ad using the determined mixed performance 

7 information. 



1 32. The method of claim 31 wherein the act scoring the ad further uses, at least, 

2 document information. 

1 33. The method of claim 31 wherein the act of scoring the ad further uses, at 

2 least, ad targeting information. 

1 34. The method of claim 31 wherein the act of scoring the ad further uses, at 

2 least, ad relevance information. 

1 35. A method comprising: 

2 a) accepting targeting function performance for a document; and 

3 b) scoring or modifying a score of each of one or more ads using the 

4 accepted targeting function performance for the document. 

1 36. The method of claim 35 wherein the ad performance includes selection and 

2 impression information. 

1 37. The method of claim 35 wherein the document is a Web page identified by a 

2 URL. 
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1 38. The method of claim 35 wherein the act of scoring includes 



2 i) selecting a scoring function using, at least, the accepted 

3 targeting function performance for the document, and 

4 ii) applying ad information and document information to the 

5 selected scoring function to generate a score. 



1 39. The method of claim 38 wherein the scoring function is a function selected 

2 from a set of functions including (A) keyword targeting, (B) document content 

3 targeting, and (C) host content targeting. 

1 40. The method of claim 35 wherein the act of scoring includes 



2 i) selecting one or more parameters of a scoring function using, at 

3 least, the accepted targeting function performance for the 

4 document, and 

5 ii) applying ad information and document information to the scoring 

6 function with the selected one or more parameters to generate a 

7 score. 



1 41. A method comprising: 

2 a) accepting targeting function performance for a set of documents; and 

3 b) scoring or modifying a score of each of one or more ads using the 

4 accepted targeting function performance for the set of documents, wherein 

5 the set of documents is a subset of a document collection. 

1 42. The method of claim 41 wherein the ad performance includes selection and 

2 impression information. 

1 43. The method of claim 41 wherein the set of documents includes related Web 

2 pages. 

1 44. The method of claim 41 wherein the act of scoring includes 
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2 i) selecting a scoring function using, at least, the accepted 

3 targeting function performance for the set of documents, and 

4 ii) applying ad information and document information to the 

5 selected scoring function to generate a score. 

1 45. The method of claim 44 wherein the scoring function is a function selected 

2 from a set of functions including (A) keyword targeting, (B) document content 

3 targeting, and (C) host content targeting. 

1 46. The method of claim 41 wherein the act of scoring includes 

2 i) selecting one or more parameters of a scoring function using, at 

3 least, the accepted targeting function performance for the set of 

4 documents, and 

5 ii) applying ad information and document information to the scoring 

6 function with the selected one or more parameters to generate a 

7 score. 

1 47. A method for determining a set of ads eligible to be served with a document, 

2 the method comprising: 

3 a) determining a first set of ads; 

4 b) accepting ad performance information for the document; 

5 c) determining a number of best performing ads for the document; and 

6 d) determining a final set of ads using the first set of ads and the number 

7 of best performing ads determined. 

1 48. A method for determining a set of ads eligible to be served with a document, 

2 the method comprising: 

3 a) determining a first set of ads; 

4 b) accepting ad performance information for a set of documents to which 

5 the document belongs, wherein the set of documents is a subset of a 

6 collection of documents; 
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7 c) determining a number of best performing ads for the set of documents; 

8 and 

9 d) determining a final set of ads using the first set of ads and the number 
10 of best performing ads determined. 

1 49. A method comprising: 

2 a) determining for a document, at least two concepts; 

3 b) determining for each of the at least two concepts, one or more ads; 

4 c) determining for each of the at least two concepts, a concept 

5 performance score; and 

6 d) updating, for at least one of the ads, an ad performance score using a 

7 concept performance score of the concept with which the ad is associated. 

1 50. The method of claim 49 wherein the act of determining a concept 

2 performance score uses document-specific ad performance scores of ads 

3 associated with the concept. 



1 51 . The method of claim 49 wherein the document belongs to a group, and 

2 wherein the act of determining a concept performance score uses 

3 group-specific ad performance scores of ads associated with the concept. 

1 52. The method of claim 51 wherein the document is a Web page and wherein 

2 the group is Web pages belonging to a Website. 

1 53. The method of claim 51 wherein the group is a cluster of related documents. 
54. The method of claim 51 wherein the group is a classification of documents. 

1 55. Apparatus comprising: 

2 a) means for tracking, for a set of one or more ads, ad set performance 

3 information with respect to a document; and 
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4 b) means for aggregating, for the document, tracked ad set performance 

5 information. 

1 56. The apparatus of claim 55 wherein the ad set performance information 

2 includes selection and impression information. 

1 57. The apparatus of claim 55 wherein the document is a Web page identified by 

2 a URL. 

1 58. The apparatus of claim 55 further comprising: 

2 c) means for thresholding the ad set performance information. 

1 59. The apparatus of claim 55 further comprising: 

2 c) means for determining a confidence measure of the ad set 

3 performance information; and 

4 d) means for combining general ad set performance information and the 

5 ad set performance information with respect to the document using the 

6 determined confidence measure. 

1 60. The apparatus of claim 59 wherein the means for determining a confidence 

2 measure use at least one of a data amount and a data age. 

1 61. Apparatus comprising: 

2 a) means for tracking, for a set of one or more ads, ad set performance 

3 information with respect to a set of documents, wherein the set of 

4 documents is a subset of a document collection; and 

5 b) means for aggregating, for the set of documents, tracked ad set 

6 performance information. 

1 62. The apparatus of claim 61 wherein the ad set performance information 

2 includes selection and impression information. 
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1 63. The apparatus of claim 61 wherein the set of documents includes related 

2 Web pages. 

1 64. The apparatus of claim 61 further comprising: 

2 c) means for thresholding the ad set performance information. 

1 65. The apparatus of claim 61 further comprising: 

2 c) means for determining a confidence measure of the ad set 

3 performance information; and 

4 d) means for combining general ad set performance information and the 

5 ad set performance information with respect to the set of documents using 

6 the determined confidence measure. 

1 66. The apparatus of claim 65 wherein the means for determining a confidence 

2 measure use at least one of a data amount and a data age. 



1 67. Apparatus comprising: 



2 a) an input for accepting, for a set of one or more ads, performance 

3 information for a document; and 

4 b) means for scoring or modifying a score of each of one or more ads 

5 using the accepted ad set performance information for the document. 



1 68. The apparatus of claim 67 wherein the ad set performance information 

2 includes selection and impression information. 

1 69. The apparatus of claim 67 wherein the document is a Web page identified by 

2 a URL. 

1 70. The apparatus of claim 67 wherein the means for scoring or modifying a 

2 scoring include: 
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3 i) means for determining a first ad score using, at least, general 

4 performance information for the ad set, and 

5 ii) means for modifying the first ad score using the accepted ad set 

6 performance information for the document. 



1 71 . The apparatus of claim 70 wherein the means for determining a first score 

2 further use, at least, document information. 

1 72. The apparatus of claim 70 wherein the means for determining a first score 

2 further use, at least, ad targeting information. 

1 73. The apparatus of claim 70 wherein the means for determining a first score 

2 further use, at least, ad relevance information. 

1 74. The apparatus of claim 70 wherein the means for scoring or modifying a 

2 scoring include: 



3 i) means for determining mixed performance information using, at 

4 least, general performance information for the ad set and the 

5 accepted ad set performance information for the document, and 

6 ii) means for scoring the ad using the determined mixed 

7 performance information. 



1 75. The apparatus of claim 74 wherein the means for scoring the ad further use, 

2 at least, document information. 

1 76. The apparatus of claim 74 wherein the means for scoring the ad further use, 

2 at least, ad targeting information. 

1 77. The apparatus of claim 74 wherein the means for scoring the ad further use, 

2 at least, ad relevance information. 



-50- 



Google-53 APP (GP-064-05-US) 
1 78. Apparatus comprising: 



2 a) an input for accepting, for a set of one or more ads, performance 

3 information for a set of documents, wherein the set of documents is a 

4 subset of a document collection; and 

5 b) means for scoring or modifying a score of each of one or more ads 

6 using the accepted ad set performance information for the set of 

7 documents. 



1 79. The apparatus of claim 78 wherein the ad set performance information 

2 includes selection and impression information. 

1 80. The apparatus of claim 78 wherein the set of documents includes related 

2 Web pages. 

1 81 . The apparatus of claim 78 wherein the means for scoring or modifying a 

2 score include: 



3 i) means for determining a first ad score using, at least, general 

4 performance information for the ad set, and 

5 ii) means for modifying the first ad score using the accepted ad set 

6 performance information for the set of documents. 



1 82. The apparatus of claim 81 wherein the means for determining a first score 

2 further use, at least, document information. 

1 83. The apparatus of claim 81 wherein the means for determining a first score 

2 further use, at least, ad targeting information. 

1 84. The apparatus of claim 81 wherein the means for determining a first score 

2 further use, at least, ad relevance information. 
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1 85. The apparatus of claim 78 wherein the means for scoring or modifying a 

2 score include: 



3 i) means for determining mixed performance information using, at 

4 least, general performance information for the ad set and the 

5 accepted ad set performance information for the set of documents, 

6 and 

7 ii) means for scoring the ad using the determined mixed 

8 performance information. 



1 86. The apparatus of claim 85 wherein the means for scoring the ad further use, 

2 at least, document information. 

1 87. The apparatus of claim 85 wherein the means for scoring the ad further use, 

2 at least, ad targeting information. 

1 88. The apparatus of claim 85 wherein the means for scoring the ad further use, 

2 at least, ad relevance information. 

1 89. Apparatus comprising: 



2 a) an input for accepting targeting function performance for a document; 

3 and 

4 b) means for scoring or modifying a score of each of one or more ads 

5 using the accepted targeting function performance for the document. 



1 90. The apparatus of claim 89 wherein the ad performance includes selection 

2 and impression information. 

1 91 . The apparatus of claim 89 wherein the document is a Web page identified by 

2 a URL. 

1 92. The apparatus of claim 89 wherein the means for scoring include 
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2 i) means for selecting a scoring function using, at least, the 

3 accepted targeting function performance for the document, and 

4 ii) means for applying ad information and document information to 

5 the selected scoring function to generate a score. 



1 93. The apparatus of claim 92 wherein the scoring function is a function selected 

2 from a set of functions including (A) keyword targeting, (B) document content 

3 targeting, and (C) host content targeting. 

1 94. The apparatus of claim 89 wherein the means for scoring include 



2 i) means for selecting one or more parameters of a scoring 

3 function using, at least, the accepted targeting function 

4 performance for the document, and 

5 ii) means for applying ad information and document information to 

6 the scoring function with the selected one or more parameters to 

7 generate a score. 

1 95. Apparatus comprising: 

2 a) an input for accepting targeting function performance for a set of 

3 documents; and 

4 b) means for scoring or modifying a score of each of one or more ads 

5 using the accepted targeting function performance for the set of 

6 documents, wherein the set of documents is a subset of a document 

7 collection. 

1 96. The apparatus of claim 95 wherein the ad performance includes selection 

2 and impression information. 



1 97. The apparatus of claim 95 wherein the set of documents includes related 

2 Web pages. 
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1 98. The apparatus of claim 95 wherein the means for scoring includes 



2 i) means for selecting a scoring function using, at least, the 

3 accepted targeting function performance for the set of documents, 

4 and 

5 ii) means for applying ad information and document information to 

6 the selected scoring function to generate a score. 



1 99. The apparatus of claim 98 wherein the scoring function is a function selected 

2 from a set of functions including (A) keyword targeting, (B) document content 

3 targeting, and (C) host content targeting. 



1 100. The apparatus of claim 95 wherein the means for scoring include 

2 i) means for selecting one or more parameters of a scoring 

3 function using, at least, the accepted targeting function 

4 performance for the set of documents, and 

5 ii) means for applying ad information and document information to 

6 the scoring function with the selected one or more parameters to 

7 generate a score. 

1 101. Apparatus for determining a set of ads eligible to be served with a 

2 document, the apparatus comprising: 

3 a) means for determining a first set of ads; 

4 b) an input for accepting ad performance information for the document; 

5 c) means for determining a number of best performing ads for the 

6 document; and 

7 d) means for determining a final set of ads using the first set of ads and 

8 the number of best performing ads determined. 

1 102. Apparatus for determining a set of ads eligible to be served with a 

2 document, the apparatus comprising: 

3 a) means for determining a first set of ads; 
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4 b) an input for accepting ad performance information for a set of 

5 documents to which the document belongs, wherein the set of documents 

6 is a subset of a collection of documents; 

7 c) means for determining a number of best performing ads for the set of 

8 documents; and 

9 d) means for determining a final set of ads using the first set of ads and 
10 the number of best performing ads determined. 

1 103. Apparatus comprising: 

2 a) means for determining for a document, at least two concepts; 

3 b) means for determining for each of the at least two concepts, one or 

4 more ads; 

5 c) means for determining for each of the at least two concepts, a concept 

6 performance score; and 

7 d) means for updating, for at least one of the ads, an ad performance 

8 score using a concept performance score of the concept with which the ad 

9 is associated. 



1 104. The apparatus of claim 103 wherein the means for determining a concept 

2 performance score use document-specific ad performance scores of ads 

3 associated with the concept. 

1 105. The apparatus of claim 103 wherein the document belongs to a group, and 

2 wherein the means for determining a concept performance score use 

3 group-specific ad performance scores of ads associated with the concept. 

1 106. The apparatus of claim 105 wherein the document is a Web page and 

2 wherein the group is Web pages belonging to a Website. 

1 1 07. The apparatus of claim 1 05 wherein the group is a cluster of related 

2 documents. 
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108. The apparatus of claim 105 wherein the group is a classification of 
documents. 
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